In the claims: 

Following is a complete set of claims as amended with this Response. 
1 . (Currently Amended) A method comprising: 

maintaining a current state of auxiliary information regarding a first frame of a 
sequence of video frames in storage in a local device hardware memory, the sequence of 
video frames being encoded as a video bit stream having video frame data for each 
respective video frame of the sequence of video frames; 

using an electronic comparator comparing the stored current state of auxiliary 
information with auxiliary information regarding a later current video frame of the 
sequence of video frames to determine differential information; and 

annotating the differential information to the video bit stream as an annotation to 
the video frame data for the current video frame only if the differential information 
indicates a change from the current state of the auxiliary information. 

2. (Original) The method of Claim 1, wherein the current state of auxiliary 
information indicates the latest encoded information in the bit stream and is maintained in 
a state storage memory. 

3. (Original) The method of Claim 1 , wherein the auxiliary information 
comprises additional non-visual information regarding video frames. 

4. (Original) The method of Claim 1, wherein the auxiliary information 
comprises video processing information regarding video frames. 

5. (Original) The method of Claim 1 , further comprising encoding the 
differential information before annotating the differential information. 
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6. (Original) The method of Claim 5, wherein the encoded differential 
information comprises a list of parameters, the parameters being described in a tag 
lookup table. 

7. (Original) The method of Claim 1, wherein annotating the differential 
information comprises extending the video bit stream format to include the differential 
information. 

8. (Canceled) 

9. (Original) The method of Claim 1, further comprising gathering the video 
information regarding the current video frame. 

10. (Original) The method of Claim 9, wherein the differential information 
comprises the difference between the current state of the auxiliary information and the 
gathered information. 

1 1 . (Previously Presented) A computer readable medium having stored 
thereon data representing instructions which, when executed by a computer, cause the 
computer to perform operations comprising: 

maintaining a current state of auxiliary information regarding a first frame of a 
sequence of video frames in storage in a local memory, the sequence of video frames 
being encoded as a video bit stream having video frame data for each respective video 
frame of the sequence of video frames; 

comparing the stored current state of auxiliary information with auxiliary 
information regarding a later current video frame of the sequence of video frames to 
determine differential information; 
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annotating the differential information to the video bit stream as an annotation to 
the video frame data for the current video frame only if the differential information 
indicates a change from the current state of the auxiliary information. 

12. (Original) The medium of Claim 11, wherein the current state of auxiliary 
information indicates the latest encoded information in the bit stream and is maintained in 
a state storage memory. 

13. (Original) The medium of Claim 11, wherein the auxiliary information 
comprises additional non-visual information regarding video frames. 

14. (Original) The medium of Claim 1 1 , further comprising encoding the 
differential information as a list of parameters, the parameters being described in a tag 
lookup table, before annotating the differential information. 

15. (Original) The medium of Claim 14, wherein annotating the differential 
information comprises extending the video bit stream format to include the differential 
information. 

16. (Previously Presented) An apparatus comprising: 

a collector to receive auxiliary information regarding a first frame of a sequence 
of video frames and to maintain in storage a current state of the auxiliary information in a 
memory of the collector, the sequence of video frames being encoded as a video bit 
stream having video frame data for each respective video frame of the sequence of video 
frames; 

a comparator to receive the auxiliary information and compare the stored current 
state of the auxiliary information with auxiliary information regarding a later current 
video frame of the sequence of video frames to determine differential information; and 
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an annotator to store the differential information in the video bit stream by 
extending the original bit stream format only if the differential information indicates a 
change from the current state of the auxiliary information. 

17. (Original) The apparatus of Claim 16, wherein the auxiliary information 
comprises additional non-visual information regarding video frames. 

18. (Original) The apparatus of Claim 16, wherein the auxiliary information 
comprises video processing information regarding video frames. 

19. (Original) The apparatus of Claim 16, wherein the collector receives the 
sequence of video frames. 

20. (Original) The apparatus of Claim 16, further comprising a state storage 
memory coupled to the collector to store the current state of the auxiliary information. 

21. (Original) The apparatus of Claim 16, further comprising an encoder to 
encode the differential information as auxiliary information for the current frame data; 

22. (Original) The apparatus of Claim 16, wherein the encoded differential 
information comprises a list of parameters, the parameters being described in a tag 
lookup table. 

23. (Previously Presented) An apparatus comprising: 

a collector to receive auxiliary information regarding a first frame of a sequence 
of video frames and to maintain in storage a current state of the auxiliary information, the 
sequence of video frames being encoded as a video bit stream having video frame data 
for each respective video frame of the sequence of video frames; 

a state storage memory coupled to the collector to store the current state of the 
auxiliary information; 
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a comparator to receive the auxiliary information and compare the stored current 
state of the auxiliary information with auxiliary information regarding a later current 
video frame of the sequence of video frames to determine differential information; and 

a tag lookup table defining settable parameters; 

an annotator to store the differential information using the settable parameters in 
the video bit stream by extending the original bit stream format only if the differential 
information indicates a change from the current state of the auxiliary information. 

24. (Original) The apparatus of Claim 16, wherein the auxiliary information 
comprises additional non-visual information regarding camera geometry and 
identification for the video frames. 

25. (Original) The apparatus of Claim 16, wherein the auxiliary information 
comprises video processing information regarding descriptions and camera positions for 
the video frames. 
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